Cameras using a single-plate CCD as an image pickup system currently constitute the mainstream of digital still cameras, video cameras and the like that are commercially marketed for general use. Such single-plate CCDs are configured with color filters disposed on the front surface, and can be classified into the two main categories of complementary color system and primary color system according to the type of color filter used.
Regardless of whether the type of color filter used is the complementary color system or primary color system, single-plate color CCDs with the configuration described as above are similar in that one color signal is assigned to one pixel. Accordingly, in order to obtain all of the color signals for one pixel, it is necessary to perform processing that interpolates color signals that have dropped out in the respective pixels.
Interpolation processing must be performed not only in such single-plate systems, but also in two-plate image pickup systems or even three-plate image pickup systems in which pixels are shifted.
Techniques in which correlations or edges are detected, and interpolation processing is performed in directions with a high correlation or flat directions with a low edge intensity, have been described (for example) in Japanese Patent Application Laid-Open No. 7-236147 and Japanese Patent Application Laid-Open No. 8-298670.
Furthermore, a technique using color correlations in local regions is described in Japanese Patent Application Laid-Open No. 2000-224601 as another example of the abovementioned interpolation processing.
Furthermore, a technique in which different interpolation methods are combined when enlargement processing is performed, e.g., in which cubic interpolation is used for R signals and G signals and nearest-neighbor interpolation is used for B signals and the like, is described in Japanese Patent Application Laid-Open No. 2000-151989.
Meanwhile, in the abovementioned Japanese Patent Application Laid-Open No. 2000-224601, a technique is described in which adaptive switching is performed between interpolation by color correlation and linear interpolation.
Means for performing interpolation in selected directions as described in the abovementioned Japanese Patent Application Laid-Open No. 7-236147 and Japanese Patent Application Laid-Open No. 8-298670 functions well in cases where the image signals are configured with a single edge structure; however, in cases where there are a plurality of edge structures as in texture images, the precision of interpolation may drop because of a failure in the selection of the direction.
Furthermore, interpolation based on color correlations such as that described in the abovementioned Japanese Patent Application Laid-Open No. 2000-224601 allows high-precision interpolation even in cases where there are a plurality of edge structures as in textures, as long as these images have a single hue. However, in the boundary regions between different hues, artifacts may be generated due to a failure in the estimation of the color correlation.
Furthermore, while means that combines different interpolation methods such as the means described in the abovementioned Japanese Patent Application Laid-Open No. 2000-151989 is advantageous in that such means allows higher-precision interpolation as a whole by performing interpolation processing in regions where these methods respectively have strong points, the question of how to perform switching control of the interpolation processing in an advantageous manner becomes important. However, since the switching method described in this Japanese Patent Application Laid-Open No. 2000-151989 is fixed, the precision drops in cases where (for example) there is a complicated edge structure in the B (blue) signal, so that the merits of the respective interpolation methods cannot be used to advantage.
Furthermore, in the technique described in the abovementioned Japanese Patent Application Laid-Open No. 2000-224601, switching control of the interpolation method is performed on the basis of an original signal in which one or more color signals drop out; accordingly, the control method is complicated, and the processing time is long.
The present invention is devised in light of the above facts; it is an object of the present invention to provide an image pickup system and image processing program which make it possible to interpolate color signals that have dropped out in image signals with a higher degree of precision.